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Executive Summary 


School quality doesn’t vary only between cities; it varies within cities. All parents know this, of course, and seek 
to enroll their children in the best schools. But parents face different scenarios in different cities. 


In some cities, it doesn’t really matter which public school one attends; variation in quality is limited. This can 
be good news if most schools are decent—the case in Arlington, Virginia, for example. But it can be bad news in 
cities where most schools are dreadful—the case in Birmingham, Alabama. For parents in those school districts, 
the only way to improve their children’s situation is to move elsewhere or enroll their kids in private school. 


In other cities, such as Washington, D.C., median quality is low, but there are high-quality exceptions. Parents in 
such school districts face the task of doing what it takes to enroll their children in the limited number of decent 
schools. And there are cities, such as San Francisco, where median quality is relatively high but where there are 
many low performers, too. Parents in those school districts should avoid having their children, in effect, draw 
the short straw. 


This paper quantifies such variation in elementary and middle school quality within 68 of the largest U.S. public 
school districts, which collectively serve about 7.8 million students. It also examines whether cities that see stu- 
dents with certain demographics (specifically, low-income and nonwhite) concentrated in certain schools expe- 
rience higher variation in school quality. The findings, especially those in the comprehensive Appendix table, can 
serve as a guide for parents, as well as an admonition to school officials in cities where quality is consistently low 
or where the quality gap between the best and worst schools is wide. 
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Introduction 


Urban public school districts in the U.S. vary in quality. Educational policies, distribution of resources, and the 
challenges that students bring to the classroom differ substantially across localities. Little surprise, then, that 
academic outcomes—such as high school graduation rates and average school performance on standardized 
tests—fluctuate widely, too. 


When comparing the quality of schools across localities, measures of average student and school performance 
are the appropriate first-order concern. But average performance doesn’t fully characterize the quality of an 
urban school system. Most students don’t attend a city’s average public school. Some students attend better 
schools, while others attend worse schools. 


That school quality varies within any particular city is well known. But previous research has not established 
the degree to which such variation in school quality—within a given school district—differs between cities. Is the 
spread between bad and good schools substantially larger in some cities than in others? This report investigates 
the question by comparing the variation in the quality of public elementary and middle schools across 68 of the 
largest public school districts in the U.S. (see Appendix for comprehensive results). 


The variation in the quality of a city’s schools (“city” and “school district” will be used interchangeably) has im- 
portant implications for parents seeking access to an appropriate school for their child, and it speaks directly to 
whether a city is providing equitable access to educational quality for residents. For instance, securing a spot at 
a decent public school is less difficult in a city where all schools perform relatively similarly (such as Arlington) 
than in a city with comparably decent average school quality but much wider variation in quality (such as San 
Francisco). 


In this paper, I find that the variation in school quality is fairly similar in America’s largest school districts, with a 
large plurality having a gap of 15—20 percentage points between good and bad schools (as defined below). Several 
cities, however, including New York and San Francisco, exhibit substantially wider school-quality gaps. 


I also examine the relationship between variation in school quality and student demographics. Overall, I find no 
strong correlation between variation in quality and the proportion of a school district’s students who are low-in- 
come or nonwhite. However, cities where low-income and nonwhite students are more heavily concentrated 
within particular schools (such as Portland, Oregon) do exhibit larger variation in quality. 
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Data and Methodology 


I measure and compare the variation in school quality 
in each of the 68 member cities of the Council for Great 
City Schools, an advocacy group, for which the nec- 
essary data are available. The focus is on these large 
school districts because each has enough schools to 
calculate meaningful measures of their quality disper- 
sion. I utilize school performance and demographic 
data developed for the purpose of ranking school per- 
formance available on the SchoolGrades.org website." 


SchoolGrades reports test-score information for 
each public elementary and middle school in the 
U.S., including traditional public, magnet, and 
charter schools.? I use an “adjusted overall pro- 
ficiency” score as my measure of school quality.’ 
Doing so allows for cross-state comparisons between 
schools and accounts for the economic profile of stu- 
dents in the school. 


When calculating an adjusted overall proficiency score, 
SchoolGrades first takes the average percentage of the 
school’s students who scored proficient, or better, on 
statewide math and reading exams. Because statewide 
tests and proficiency standards vary, SchoolGrades 
adjusts a school’s score to align it with a uniform na- 
tional standard. To do this, it takes the difference 
between the average percentage of students who score 
proficient, or better, on the respective statewide exam 
and the percentage scoring proficient, or better, on the 
National Assessment of Educational Progress (NAEP), 
a test given by the federal government to representa- 
tive samples of students in each state. 


When assessing a school’s performance, it is important 
to account for the socioeconomic profile of its students. 
Lower-income students face challenges that affect 
their academic performance but are independent of 
the school’s quality. To account for this, SchoolGrades 
adjusts each school’s score to represent the predicted 
percentage of students who would score proficient, or 
better (according to NAEP standards), if the school had 
the same proportion of students eligible for free, or re- 
duced-price, lunch—a common measure for determin- 
ing whether a student is low-income—as the average 
U.S. school. (Adjusting for students’ socioeconomic 
background ends up increasing the quality score for 
schools with large low-income populations, and it ends 
up reducing the score for schools that serve primarily 
higher-income students.) 


The SchoolGrades scores that I use are, of course, an 
imperfect proxy for a school’s quality: we ask schools 
to do more than increase math and reading test scores, 
and there may well be other factors that are outside 


a school’s control but nevertheless affect test scores. 
Because of these limitations, parents and policymakers 
should not use the SchoolGrades scores, or any other 
school evaluation measure, in isolation to make en- 
rollment or policy decisions. Nonetheless, the measure 
provides a meaningful approximation for comparing 
school performance—holding constant the challenges 
that students bring to the school—across states. 


I also utilized SchoolGrades data on the percentage of 
students in each school who are eligible for free, or re- 
duced-price, lunch, supplementing such numbers with 
U.S. Department of Education data on the number and 
proportion of a school’s students who are nonwhite. 
I also calculated a Gini coefficient to measure the dis- 
persion of low-income and nonwhite students across a 
city’s schools.* 


Analysis 


Figure 1 illustrates two of the scenarios described 
in the executive summary—scenario 2: low average 
quality, low variation; and scenario 3: low average 
quality, high variation. Blue squares represent schools 
in the Santa Ana, California, school district. Red trian- 
gles represent schools in the Washington, D.C., school 
district. The vertical axis represents the adjusted per- 
centage of students in a school who are proficient or 
better. The horizontal axis represents a school’s profi- 
ciency rank within its district. The dashed red and blue 
lines show the 25th and 75th percentiles for school pro- 
ficiency in the respective cities (henceforth, a proxy for 
“good” and “bad” schools within a district). The vari- 
ation in school quality in each district is measured by 
the gap between the district’s 25th and 75th percentile 
lines.° 


Overall, school performance is quite similar in the two 
cities. About 29% of students in the average D.C. school 
score proficient, or better, compared with about 25% 
of students in the average Santa Ana school. In both 
school districts, about 23% of students score proficient, 
or above, in the median school (the 50th percentile). 


But the variation in school quality within these districts 
differs meaningfully. In Santa Ana, in the 75th percen- 
tile school, the share of students who score proficient, 
or better, is only 9 percentage points higher than in 
the 25th percentile school. Yet in D.C., the “proficiency 
gap” is 26 percentage points. 


Given that the average performance and the median 
performance in the two cities are so similar—and 
that nearly all parents want to send their kids to as 
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Variation in School Quality: Santa Ana, 


California vs. Washington, D.C. 
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Difference in % of Students Proficient, 


Schools in 75th and 25th Percentiles 


Source: Author’s calculations 


high-quality a school as_ possible— 
should parents and policymakers prefer 
the school-quality distribution in D.C. or 
in Santa Ana? The answer depends on 
how one weighs the trade-off between 
the chance to attend a high-performing 
school against the risk of ending up at a 
low-performing school. 


Economic theory suggests that when 
choosing between two lotteries with the 
same average (i.e., expected value), a 
person who is at all wary of risk would 
prefer Santa Ana because it exhibits the 
smaller variation in potential values. But 
the situation is more complicated when 
comparing districts with a wider gap in 
the performance of the average school. 
And it is complicated further when we 
consider that not all parents have the 
same access to better schools in a city. 


How policymakers and parents should 
weigh the average and spread of school 
performance in a city is an issue for 
further research. But the first step in 
any such analysis is to assess the extent 
to which the variation in school quality 
differs across cities. 


The dots on Figure 2 illustrate the 
difference in the adjusted percentage 
proficient at the 75th and 25th percen- 
tiles for all 68 school districts. Cities 
that exhibit the least variation in school 
quality are highest on the vertical axis. 
In 44% of cities (30 of 68), the difference 
in the quality score for the 75th and 25th 
percentile schools is 15-20 percentage 
points. In 12 cities, the quality gap is 
less than 15 percentage points. And in 
11 cities, it is more than 25 percentage 
points. 


Variation in school quality is greatest in 
cities such as New York, San Francisco, 
and Newark (NJ), where the proficiency 
gap exceeds 25 percentage points. 
Variation in school quality is lowest in 
cities such as Birmingham, Rochester 
(NY), and Santa Ana, where the 
proficiency gap is less than 10 percentage 
points. 
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Figure 3 illustrates the relationship between a city’s 
variation in quality and the quality of its median school. 
The 68 circles represent cities—the bigger the circle, 
the greater the number of schools in the city. The verti- 
cal axis is the adjusted percentage of students who are 
proficient, or better, at the city’s median school. The 
horizontal axis is the difference in the adjusted per- 
centage of students proficient between the 75th and 
25th percentile schools. 


The vertical and horizontal red lines separate the cities 
into four quadrants that demonstrate their position 
relative to the medians—for both variation in quality 
and median quality—for the 68 cities. The 15 in the 
top left quadrant have relatively high median school 
quality but relatively low variation in school quality. 
The 15 in the bottom right quadrant have high varia- 
tion in quality but low median quality. The 19 in the 
top right quadrant have both relatively high median 
quality and high variation in quality. And the 19 in the 
bottom left quadrant have both low variation in quality 
and low median quality. 


The dashed regression line illustrates a positive cor- 
relation such that, on average, cities with higher vari- 
FIGURE 3. 


Variation in School Quality and Median 
School Quality 


ation in school quality tend to have higher median 
quality.® But the relationship between the median and 
the spread of school quality is not deterministic—some 
school districts with relatively high median quality 
experience lower variation in quality, and vice versa. 
The relationship between median quality and spread 
of quality is also nonlinear. The relationship is steepest 
for school districts with low median proficiency,’ and 
it flattens considerably for better school districts, such 
that there is little relationship starting for districts with 
about 30% of students scoring proficient or better. If 
the four cities with the lowest quality variation (Bir- 
mingham; Rochester; Santa Ana; and Aurora, Colora- 
do) are excluded, there is not even a clear relationship 
between variation and median quality. 


Figure 3 also shows that school-quality variation is not 
determined by the number of schools in a city. Small, 
medium, and large cities are found in all quadrants. 


What factors might account for the large gap in school 

quality between different cities? Data limitations re- 

strict our ability to answer this question. For instance, 

the large number of selective traditional public schools 

in New York City likely contributes to its status as the 
major school district with the largest 
difference between the 75th and 25th 
percentile schools. But we can’t analyze 
this particular factor because not all 68 
cities report such data. 


There are, however, data for all 68 
cities on two demographic factors: the 
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O below is descriptive, not causal—i.e., it 
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certain variations in school quality.)* 


New York City 


Are school districts with higher pro- 
portions of low-income and nonwhite 
students more likely to have a wider 
variation in school quality? Figure 4 
illustrates the relationship between 
school-quality variation (y-axis) and 
the proportion of students who are el- 
igible for free, or reduced-price, lunch 
(x-axis). Figure 5 does the same for 
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30 nonwhite students. (In both figures, 
dots represent cities while the sloping 
red line is a regression line that illus- 
trates the relationship between the 
two variables.) Though the variation 
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Variation in School Quality and Proportion 
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Variation in School Quality and the 
Distribution of Low-Income Students 
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in school quality tends to be lower as the proportion 
of low-income and nonwhite students rises, the rela- 
tionship is not strong, as indicated by the dispersion 
of dots.° 


In addition to differences in the proportion of non- 
white and low-income students, the extent to which 
such students are distributed across schools varies 
across cities. School districts where low-income and 
minority students are heavily concentrated in specific 
schools might exhibit larger variation in school quality, 
for at least two reasons. 


First, cities that do not equally distribute such students 
might also inequitably allocate resources between 
schools—directly, via spending per pupil, and indirect- 
ly, via, say, allocating better or worse teachers. Second, 
low-income students who attend schools with heavier 
concentrations of higher-income students often have 
access to more high-performing peers, which can have 
a positive impact on academic outcomes. 


Figures 6 and 7 illustrate the relationship between 
school-quality variation in a city and the Gini coeffi- 
cient for the distribution of, respectively, low-income 
and nonwhite students in a city. (The higher the Gini 
coefficient, the greater the student concentration in 
specific schools.) Both figures show that cities where 
low-income (such as Minneapolis) or nonwhite stu- 
dents (such as Pinellas County, Florida) are heavily 
concentrated in specific schools tend to exhibit greater 
variation in school quality." 


Conclusion 


In 44% of America’s largest school districts (30 of 68), 
the quality gap between schools at the 75th and 25th 
percentiles was 15—20 percentage points. However, in 
18% of cities (12 of 68), the quality gap was less than 15 
percentage points. And in 16% of cities (10 of 68), the 
gap was more than 25 percentage points. 


As for the relationship between variation in school 
quality and a district’s proportion of students who are 
low-income or nonwhite, no strong correlation exists. 
However, cities where low-income and nonwhite stu- 
dents are more heavily concentrated within particular 
schools do exhibit a larger variation in school quality. 


More broadly, the Appendix shows that overall school 
quality in most large U.S. public school districts 
remains dismal. In only nine districts (13% of the total) 
were at least 40% of students proficient (after adjust- 
ing for family income), or better, at the median school; 
and in a mere two districts, more than 50% of students 
were proficient, or better, at the median school. Of 
these nine districts, just three had a variation in school 
quality of less than 20 percentage points—and none 
had a variation of less than 15 percentage points. 


In short, few of America’s largest public school dis- 
tricts offer consistently decent elementary and middle 
schools. But motivated parents in low-quality districts 
should not despair entirely: in 38 districts (43% of 
the total), at least 40% of students were proficient, or 
better, at the 75th percentile school. In other words, 
in many school districts that are of low quality overall, 
there are pockets of decent schools. 
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Appendix 


School 
District 


New York, NY 
Los Angeles, CA 
Chicago, IL 

Dade County, FL 
Clark County, NV 
Houston, TX 
Philadelphia, PA 
LEED 

Broward County, FL 
Dallas, TX 
Hillsborough, FL 
San Diego, CA 
Orange, CA 

Palm Beach, FL 
Baltimore, MD 
Charlotte, NC 
Duval County, FL 
Albuquerque, NM 


Washington, DC 


Columbus, OH 
Detroit, Ml 

Denver, CO 
Milwaukee, WI 
Austin, TX 
Jefferson County, KY 
Fort Worth, TX 
Cleveland, OH 
Pinellas County, FL 
Guilford, NC 
Oakland, CA 
Boston, MA 

San Antonio, TX 


Fresno, CA 
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Charleston County, SC 29 31.8 24.7 37.6 12.9 71.6 40.5 18.4 27.8 
Providence, RI 29 23.7 16.8 32.1 i168) 90.5 82.1 22.2 20.8 


Source: Author's calculations 
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Endnotes 


SchoolGrades.org was developed jointly by the Manhattan Institute and GreatSchools. Using the latest data on school performance from GreatSchools, 
MI developed a methodology to standardize test results across the 50 states while accounting for differences in students’ socioeconomic backgrounds. 


| exclude vocational schools and other specialty schools from the sample because such schools focus less on test scores and more on directly training 
students for jobs. 


“Our Grades,” SchoolGrades.org. | present results from analyses that do not weight for school enrollment. Weighting for enrollment has no impact on 
the overall results and makes only a marginal difference for select districts. 


When considering the dispersion of nonwhite students, a Gini coefficient of O would indicate that nonwhite students are evenly represented in all the 
city’s schools; a Gini coefficient of 100 would indicate that all nonwhite students in the city are enrolled in a single school. 


| use this measure because it is understandable for a wide range of readers. Results are similar but not identical when | instead use the standard 
deviation of school performance as the measure of dispersion. 


The Pearson correlation coefficient is 0.4676. 


Some of this relationship at the bottom of the distribution is likely mechanical—if a district has a very low median school quality, there is a lower 
bound on the quality of the 25th percentile school, thus limiting the potential for quality variation. 

A causal analysis would require holding constant all factors that are associated with the variation in school quality and with the factors of interest here. 
Such an analysis is not possible, given data limitations. 


Simple bivariate correlations find coefficients of -0.15 and -0.26 for the relationship between the measure of variation and the proportion of 
lower-income and nonwhite students, respectively. 


10 For both characteristics, the correlation coefficient for the relationship with the measure of variation is 0.44. 
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